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Abstract: Let (W, H, /i) be the classical Wiener space, assume that U\ = Iw + u \ is an adapted pertur- 
bation of identity where the perturbation u\ is an //-valued map, defined up to ^-equivalence classes, such 
that its Lebesgue density s — > u\(s) is almost surely adapted to the canonical filtration of the Wiener space 
and depending measurably on a real parameter A. Assuming some regularity for u\, its Sobolev derivative 
and integrability of the divergence of the resolvent operator of its Sobolev derivative, we prove the almost 
sure and //-regularity w.r. to A of the estimation E[ii\(s)\U\(s)] and more generally of the conditional ex- 
pectations of the type E[F \ U\(s)j for nice Wiener functionals, where (U\(s), s £ [0, 1]) is the the filtration 
which is generated by U\. These results are applied to prove the invertibility of the adapted perturbations 
of identity, hence to prove the strong existence and uniqueness of functional SDE's; convexity of the entropy 
and the quadratic estimation error and finally to the information theory. 
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1. Introduction 

The Malliavin calculus studies the regularity of the laws of the random variables (functionals) defined 
on a Winer space (abstract or classical) with values in finite dimensional Euclidean spaces (more 
generally manifolds) using a variational calculus in the direction of the underlying quasi-invariance 
space, called the Cameron-Martin space. Although its efficiency is globally recognized by now, for 
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the maps taking values in the infinite dimensional spaces the Malliavin calculus does not apply 
as easily as in the finite dimensional case due to the absence of the Lebesgue measure and even 
the problem itself needs to be defined. For instance, there is a notion called signal to noise ratio 
which finds its roots in engineering which requires regularity of infinite dimensional objects with 
respect to finite dimensional parameters (cf.[U [8j [9j [101 HU H2])- Let us explain the problem along 
its general lines briefly: imagine a communication channel of the form y — y/Xx + w, where x 
denotes the emitted signal and w is a noise which corrupts the communications. The problem of 
estimation of the signal x from the data generated y is studied since the early beginnings of the 
electrical engineering. One of the main problems dealt with is the behavior of the L 2 -error of the 
estimation w.r. to the signal to noise ratio A. This requires elementary probability when x and 
w are independent finite dimensional variables, though it gives important results for engineers. In 
particular, it has been recently realized that (cf. [8 ( 22 ), in this linear model with w being Gaussian, 
the derivative of the mutual information between x and y w.r. to A equals to the half of the mean 
quadratic error of estimation. The infinite dimensional case is more tricky and requires already the 
techniques of Wiener space analysis and the Malliavin calculus (cf. [12]). The situation is much more 
complicated in the case where the signal is correlated to the noise; in fact we need the A-regularity 
of the conditional expectations w. r. to the filtration generated by y, which is, at first sight, clearly 
outside the scope of the Malliavin calculus. 

In this paper we study the generalization of the problem mentioned above. Namely assume that 
we are given, in the setting of a classical Wiener space, denoted as (W,H,n), a signal which is of 
the form of an adapted perturbation of identity: 



where (Wt,t G [0,1]) is the canonical Wiener process, ii\ is an element of L 2 (ds x dfj) which is 
adapted to the Brownian filtration ds-almost surely and A is a real parameter. Let U\{t) be the sigma 
algebra generated by (U\(s), s < t). What can we say about the regularity, i.e., continuity and/or 
differentiability w.r. to A, of the functionals of the form A — »• E[F | U\{t)] and A — > E[F \ U\ = w] 
(the latter denotes the disintegration) given various regularity assumptions about the map A — > 
like differentiability of it or its ii-Sobolev derivatives w.r. to A? We prove that the answer to these 
questions depend essentially on the behavior of the random resolvent operator (Ih + Vitx) , where 
Vua denotes the Sobolev derivative of u\, which is a quasi- nilpotent Hilbert-Schmidt operator, hence 
its resolvent exists always. More precisely we prove that if the functional 



is in L 1 (c?A x dfi, [0, M] x W) for some M > 0, where 5 denotes the Gaussian divergence and p(—5u) 
is the Girsanov-Wick exponential corresponding to the stochastic integral 6u (cf. the next section) , 
then the map A — > L\ is absolutely continuous almost surely where L\ is the Radon- Nikodym 
derivative of U\\i w.r. to fj, and we can calculate its derivative explicitly. This observation follows 
from some variational calculus and from the Malliavin calculus. The iteration of the hypothesis 
(|l.ip by replacing 5{{Ir +^u\)~ 1 j\ u x) with its A-derivatives permits us to prove the higher order 
differentiability of the above conditional expectations w.r. to A and these results are exposed in 
Section [3] In Section [4j we give applications of these results to show the almost sure invertibility of 



U\(t,w) =W t {w) + u\(s,w)ds, 
Jo 




(1.1) 
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the adapted perturbations of the identity, which is equivalent to the strong existence and uniqueness 
results of the (functional) stochastic differential equations. In Section [SJ we apply the results of 
Section[3]to calculate the derivatives of the relative entropy of U\[i w.r. to fi in the general case, i.e., 
we do not suppose the a.s. invertibility of U\, which demands the calculation of the derivatives of the 
non-trivial conditional expectations. Some results are also given for the derivative of the quadratic 
error in the case of anticipative estimation as well as the relations to the Monge-Kantorovich measure 
transportation theory and the Monge- Ampere equation. In Section [6] we generalize the celebrated 
result about the relation between the mutual information and the mean quadratic error (cf. [Tl[9lll0]) 
in the following way: we suppress the hypothesis of independence between the signal and the noise 
as well as the almost sure invertibility of the observation for fixed exterior parameter of the signal. 
With the help of the results of Section [3] the calculations of the first and second order derivatives 
of the mutual information w.r. to the ratio parameter A are also given. 

2. Preliminaries and notation 

Let W be the classical Wiener space C([0,T],1R™) with the Wiener measure [i. The corresponding 
Cameron-Martin space is denoted by H . Recall that the injection H W is compact and its 
adjoint is the natural injection W* ^ H* C L 2 (/i). Since the image of \i under the mappings 
w w + h, h E H is equivalent to fi, the Gateaux derivative in the H direction of the random 
variables is a closable operator on L p (^i)-spaces and this closure is denoted by V and called the 
Sobolev derivative (on the Wiener space) cf., for example (T3] [14]. The corresponding Sobolev 
spaces consisting of (the equivalence classes) of real-valued random variables will be denoted as 
Ep^, where k <E IN is the order of differentiability and p > 1 is the order of integrability. If 
the random variables are with values in some separable Hilbert space, say $, then we shall define 
similarly the corresponding Sobolev spaces and they are denoted as ID Pi fc($), p > 1, k G 1ST. Since 

V : IDp.fe — > D Pi fc_i(i?) is a continuous and linear operator its adjoint is a well-defined operator 
which we represent by 6. A very important feature in the theory is that 5 coincides with the Ito 
integral of the Lebesgue density of the adapted elements of TD Pi k{H) (cf. [T31 HI] ). 

For any t > and measurable / : W — > IR+, we note by 

Ptf(x) = f f (e-*x + Vl ~ e- 2t y) fx(dy) , 
Jw v ' 

it is well-known that (Pt,t € H+) is a hypercontractive semigroup on L p (/i),p > 1, which is called 

the Ornstein-Uhlenbeck semigroup (cf. [TS] H3]) • ^ s infinitesimal generator is denoted by — C and we 

call C the Ornstein-Uhlenbeck operator (sometimes called the number operator by the physicists). 

The norms defined by 

(2-2) ||0|| M = ||(/ + £)^ 2 0|| LP(Al) 

are equivalent to the norms defined by the iterates of the Sobolev derivative V. This observation 
permits us to identify the duals of the space D Pi fc($);p > 1, k € IN by ID 9i _ / t($'), with q^ 1 = 1— 
where the latter space is defined by replacing k in (|2.2j) by — k, this gives us the distribution spaces 
on the Wiener space W (in fact we can take as k any real number). An easy calculation shows 
that, formally, 6 o V = C, and this permits us to extend the divergence and the derivative operators 
to the distributions as linear, continuous operators. In fact 8 : lD q k(H ® $) — > ID g / c _i($) and 

V : ID (?! fc(<I') — > TDq k-iiH ® $) continuously, for any q > 1 and fceE, where H ® $ denotes the 
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completed Hilbcrt-Schmidt tensor product (cf., for instance [T5J [T31 HH])- We shall denote by ID(<&) 
and ©'($) respectively the sets 

D(3)= f) E> p , fc ($), 
p>i,feeiN 

and 

E»'($)= |J D Pl _ fc (#) , 

p>l : keTN 

where the former is equipped with the projective and the latter is equipped with the inductive limit 
topologies. 

Let us denote by (Wt, t £ [0, 1]) the coordinate map on W which is the canonical Brownian motion 
(or Wiener process) under the Wiener measure, let (J-t,t g [0,1]) be its completed filtration. The 
elements of L 2 (fi,H) = W)2,o(H) such that w —> u(s,w) are ds-a.s. Js measurable will be noted 
as L 2 (/j,,H) or T>2 {H). L°(^,7J) is defined similarly (under the convergence in probability). Let 
U : W — > W be defined as U — Iw + u with some u € L|J(/i, H), we say that U is /i-almost surely 
invertible if there exists some V : W — > W such that V/j, <C /i and that 

fi {w : U o =7o U{w) = w} = 1 . 

The following results are proved with various extensions in [TBI E] : 

Theorem 1. Assume that u e L®(p,H), let L be the Radon- Nikodym density of Ufi = (Iw + u)fi 
w.r. to [i, where U/i denotes the image (push forward) of [i under the map U. Then we have 

(1) 

E[LlogL] < ^\\u\\ 2 L2 ^ H) = l -E J \u s \ 2 ds. 

(2) Assume that E[p(—5u)\ — 1, then we have the equality: 

(2-3) E[L\ogL]= l -\\u\\l 2 ^ H) 

if and only if U is almost surely invertible and its inverse can be written as V — Iw + v, 
with v G L°(n, H). 

(3) Assume that E[L\ogL — logL] < oo and the equality i2.3\) holds, then U is again almost 
surely invertible and its inverse can be written as V — Iw + v, with v € L®(/jL, H). 

The following result gives the relation between the entropy and the estimation ( cf. [15] for the 
proof): 

Theorem 2. Assume that u G L 2 (p,H), let L be the Radon-Nikodym density of Ufi = (Iw + u)fi 
w.r. to (i, where Ufi denotes the image (push forward) of jj, under the map U and let (Ut,t £ [0,1]) 
be the filtration generated by (t,w) — > U(t,w). Assume that E[p(— Su)] — 1. Then we have 

i r 1 

E[L log L] - -E / \E[u s \U s ]\ 2 ds. 
* Jo 

Loll E[p(-8u)\U] = 1 

^-almost surely. 
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3. Basic results 

Let {W,H,p) be the classical Wiener space, i.e., W = C o ([0, 1], H d ), H = H l {[Q, 1], H d ) and p is 
the Wiener measure under which the evaluation map at t £ [0, 1] is a Brownian motion. Assume 
that U\ : W -> W is defined as 

U x (t,w) =W t (w) + [ ux(s lW )ds, 
Jo 

with A € IR being a parameter. We assume that u\ € L 2 a ([Q, 1] x W, dt x d/i), where the subscript " Q " 
means that it is adapted to the canonical filtration for almost all s G [0, 1]. We denote the primitive 
of u\ by u\ and assume that E[p(—5ux)} = 1, where p denotes the Girsanov exponential: 

p(-Sux) = cxp ^- j u\(s)dW s - ^ y |u A (s)| 2 ds^ . 

We shall assume that the map A — > «a is differentiable as a map in L„([0, 1] x W, x dp,), we denote 
its derivative w.r. to A by u' x (s) or by u'(X, s) and its primitive w.r. to s is denoted as u' x (t). 

Theorem 3. Suppose that A — >• ua € L^ oc (IR, dA; D Pi i(if)) /or some p > 1, with E\p{—8u\)] = 1 
/or any A > and also that 

E f {1 + p{-6u a ))\E[6{K a u' a )\U a ]\ p da <oo, 
Jo 

where K a = [In + Vti Q ) _1 . Then the map 

dp 

is absolutely continuous and we have 

da . 

to 

Proof: Let us note first that the map (A, w) — > L\(w) is measurable thanks to the Radon-Nikodym 
theorem. Besides, for any (smooth) cylindrical function /, we have 

^E[f°Ux] = E[(\7foUx,u' x ) H ] 

= E[((I H +Vux)- 1 *V(foUx),u' x ) H } 

= E[(\7(f oU x ),(Ih + Vu x )- 1 u' x ) h ] 

= EifoUxSiilH + Vuxr'u'x}} 

= E[foU x E[S(K x u' x )\U x }} 

= E[fE[6(Kxu x )\Ux = w]Lx}. 

Hence, for any fixed /, we get 

^(f,Lx) = (f,L x E[S(Kxu' x )\Ux = w}) , 

both sides of the above equality are continuous w.r. to A, hence we get 

r x 

< f,L x > - < f,L >= 



Lx(w) = L exp / E S(K a u' a )\U a = w 
Jo 1 



/ <f,L a E[S(K a u' a )\U a =w]>da. 
Jo 
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From the hypothesis, we have 



rX 

El L a \E[S(K a u' a )\U a = w]\da = E I \E[S(K a u' a )\U a ]\ da < oo . 



o 



By the measurability of the disintegrations, the mapping (a,w) — > E[5{K a u' a )\U a — w] has a 
measurable modification, hence the following integral equation holds in the ordinary sense for almost 
all w € W 



L X = L + f L a E[5{K a u' a )\U a = w]da , 
Jo 



for A > 0. Therefore the map A — > L\ is almost surely absolutely continuous w.r. to the 
Lebesgue measure. To show its representation as an exponential, we need to show that the map 
a — > E[5{K a u' a )\U a = w] is almost surely locally integrable. To achieve this it suffices to observe 
that 

e[ \E[6(K a u' a )\U a = w]\da = E [ \E[S(K a u' a )\U a = w]\^-da 

JO Jo 

= E f \E[S{K a u' a )\U a ]\ 1 TT da 

Jo L a ° U a 



= El \E[d{K a u' a )\U a ]\E[p(-Su a )\U a ]da < oo 
Jo 

by hypothesis and by Theorem [2] Consequently we have the explicit expression for L\ given as: 



L\{w) = Loexp / E[S(K a u' a )\U a = w]da. 
Jo 



□ 



Remark 1. An important tool to control the hypothesis of Theorem\3\is the inequality of T. Carleman 
which says that (cf. [3], Corollary XI. 6. 28) 

\\&et 2 {I H + A){I H +A)- 1 \\ <expi(||v4||2 + l) , 

for any Hilbert- Schmidt operator A, where the left hand side is the operator norm, det2(/ff + A) 
denotes the modified Carleman- Fredholm determinant and \\ ■ W2 denotes the Hilbert- Schmidt norm. 
Let us remark that if A is a quasi-nilpotent operator, i.e., if the spectrum of A consists of zero only, 
then det2(/f/ + A) — 1, hence in this case the Carleman inequality reads 

||(I H + A)-i<expi(|H|2 + l) . 

This case happens when A is equal to the Sobolev derivative of some u € lD Pj i(i?) whose drift u is 
adapted to the filtration (J- t , t £ [0, 1]), 

From now on, for the sake of technical simplicity we shall assume that u\ is essentially bounded 
uniformly w.r. to A. 

Proposition 1. Let F G L p (p) then the map A — » E[F\U\ = w] is weakly continuous with values 
in LP-(nf^. 



p— denotes any p' < p and q+ any q' > q 
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Proof: First we have 



/ \E[F\U x = w]\Vd j i = f \E[F\U x =w]\P^d^ 

JW JW -"A 

\E[F\U x \r -±—dn 



11 L\oU\ 

\E[F\Ux]\ p E[p(-Su x )\UxW 
hence E[F\U\ = w] € L p ~(^) for any L G Besides, for any / G C 6 (W), 

£[/ o U x F] = E[fE[F\U x = w] L x ] 

therefore 

\E[foU x F}\ < \\F\\ p \\foUx\\ q <C q \\F\\ p \\\\f\\ q+ . 

This relation, combined with the continuity of A — > f o U\, due to the Lusin theorem, in L q for any 
/ G L q+ . implies the weak continuity of the map A — > [F\U\ — w]L x with values in L p ~(/j,), since 
A — > L> and A — > (La) -1 are almost surely and strongly continuous in L p (/z), the claim follows. □ 



Theorem 4. Assume that F G IDp,i /or some p > 1 and that 

E [ \S(FK a u' a )\da < oo 
Jo 

for any A > 0, then A — ¥ E[F\U X — w] is fi-a.s. absolutely continuous w.r. to the Lebesgue measure 
dX and the map A — > B[i^|LL\] is almost surely and hence LP -continuous. 

Proof: Using the same method as in the proof of Theorem [3l we obtain 

JLe[6oU x F] = ^-E[9E[F\U x = w]L x ] 
dX dX 

= E[6L x E[6(FKxu'x)\Ux = w]] 
for any cylindrical function 9. By continuity w.r. to A, we get 

E 9(l x E[F\Ux = w]- LqE[F\U =w]} = E[9L a E[8{FK a u' a )\U a = w]]da . 
By the hypothesis 

E [ \L a E[5{FK a u' a )\U a = w}\da < oo 
Jo 

and since 9 is an arbitrary cylindrical function, we obtain the identity 

L X E[F\U X = w] - L E[F\U a = w] = [ L a E[S(FK a u' a )\U a = w]da 

Jo 

almost surely and this proves the first part of the theorem since A — > L\ is already absolutely 
continuous and strictly positive. For the second part, we denote E[F\U\] by F(X) and we assume 
that (A„,n > 1) tends to some A, then there exists a sub-sequence (F(Xk t ),l > 1) which converges 
weakly to some limit; but, from the first part of the proof, we know that (E[F\Ux k[ = w],l > 1) 
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converges almost surely to E[F\U X = w] and by the uniform integrability, there is also strong 
convergence in L p ^{p). Hence, for any cylindrical function G, we have 

E[F(\ kl )G] = E{E{F\U Xki =w}E{G\U Xki =w]L Xki } 
-> E[E[F\U x =w]E[G\U x =w]L x ] 
= E[F(X)G}. 

Consequently, the map A — > F(X) is weakly continuous in L p , therefore it is also strongly continuous. 

□ 

Remark: Another proof consists of remarking that 

E[F\U x = w]\ w=Ux =E[F\U x ] 

/i-a.s. and that A — > E[F\U\ = w] is continuous a.s. and in L p ~ from the first part of the proof and 
that (La, A e [a, b]) is uniformly integrable. These observations, combined with the Lusin's theorem 
imply the continuity in L°(fj,) (i.e., in probability) of A — > E[F\U\] and the L p -continuity follows. 

We shall need some technical results, to begin with, let U x denote the shift defined on W by 

/•■At 

U£(w) =w+ / it\(s)ds, 
Jo 

for t e [0, 1]. We shall denote by L\(t) the Radon-Nikodym density 

= Lx(r). 



dfj, 

Lemma 1. We have the relation 

L x (r) = E[L X \F T ] 

almost surely. 

Proof: Let / be an Jv-measurable, positive, cylindrical function; then it is straightforward to see 
that / o U\ = f o U x , hence 

E[fL x ] = E[foU x ] = E[foU T x ] = E[fL x (r)} . 

□ 

Lemma 2. LetU x {t) be the sigma algebra generated by {U x (s); s < t}. Then, we have 

E[f\Ul{l)]=E[f\U{] 
for any positive, measurable function on W . 

Proof: Here, of course the second conditional expectation is to be understood w.r. to the sigma 
algebra generated by the mapping and once this point is fixed the claim is trivial. □ 
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Proposition 2. With the notations explained above, we have 

L x {r) = L (t) cxp f E[S{(I H + Vul)- l vZ}\t% = w]da . 
Jo 

Moreover, the map (A,t) — > L\(t) is continuous on Hi x [0, 1] with values in L p (/i) for any p > 1. 

Proof: The first claim can be proved as we have done in the first part of the proof of Theorem [3] 
For the second part, let / be a positive, measurable function on W; we have 

E[foUZ] = E[fL x (T)}. 

If (t„, A„) — > (t, A), from the Lusin theorem and the uniform integrability of the densities (L Xn (r n ), n > 
1), the sequence (/ o U x n , n > 1) converges in probability to / o U x , hence, again by the uniform 
integrability, for any q > 1 and / g L q {jj), 

HmE[fL Xn (T n )] = E[fL x (r)]. 

n 

From Lemma [TJ we have 

E[L Xn (T n ) 2 } = E[L\ n (t„) E[L\ n \J- Trl ]] 
- E[L Xn {r n )L Xn ], 

since, from Theorem [3j L\ n — > L\ strongly in all L p -spaces, it follows that (A,r) — > L\(t) is 
£ 2 -continuous, hence also L p -continuous for any p > 1. □ 



Proposition 3. The mapping (A,r) — > L x (t) is a.s. continuous, moreover the map 

(t,w) (A ->• L\(t, w)) 

is a C(IR) -valued continuous martingale and its restriction to compact intervals (of X) is uniformly 
integrable. 

Proof: Let us take the interval A € [0, T], from Lemma[T]we have L x (t) — E[L x \JFt], since C([0, T]) 
is a separable Banach space and since we are working with the completed Brownian filtration, the 
latter equality implies an a.s. continuous, C([0, T])-valued uniformly integrable martingale. □ 



Theorem 5. Assume that 



JO 



E / (\S(u a {s)K a u' a )\ + K(s)| 2 ) ds < 



50 



for any A > 0, then the map 

X^E[u x {t)\U x (t)} 
is continuous with values in L p (p, L 2 ([0, l],lR d )) 7 p > 1. 

Proof: Let £ € L^°(/i,_ff) be smooth and cylindrical, then, by similar calculations as in the proof 
of Theorem [4j we get 

-^E[(£oU x ,u x ) H ] = ^<C°^,ka >=^<?o(/ A ,« A > 

= E [ i s L x (s)E [S(u x (s)K x u' x ) + u' x (s)\Ul =w]ds, 
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but the l.h.s. is equal to 

£7[(V£ o U x [u' x },u x ) H + (£ ° U x ,u' x ) H ] , 

which is continuous w.r. to A provided that £ is smooth, and that A — > (u' a ,ua) is continuous in i p 
for p > 2. Consequently, we have the relation 

< C° U x ,u x > - < £o U ,u >= £ / / £ s L a (s).E[<S(u a (s)ir a t4) +u' a (s)|y* = w]dsda 

Jo Jo 

and the hypothesis implies that A — > L^(s)-B[u>(s)|i7| = io] is /i-a.s. absolutely continuous w.r. to 
the Lebesgue measure dA. Since A — > £a(s) is also a.s. absolutely continuous, it follows that 
A — > ^[ua(s)|C/| = w] is a.s. absolutely continuous. Let us denote this disintegration as the kernel 
N x (w,ii x (s)), then 

N x (U s x (w),u x (s)) = E[u x (s)\U s x ] 

a.s. From the Lusin theorem, it follows that the map A — » N x (U^,ii x (s)) is continuous with values 
in i 2 ([0, 1], H d )) and the L p -continuity follows from the dominated convergence theorem. □ 



Remark 2. In the proof above we have the following result: assume that A — > /a is continuous in 
L°(/i), t/ien A — > /a o [7 A is a/so continuous in L (/j,) provided that the family 

is uniformly integrable for any compact interval [a, b] . To see this, it suffices to verify the sequential 
continuity; hence assume that A„ — > A, then we have 

m{|/a„ o U Xn - f x o U X \ > c} < M {|/a„ o U Xn - f x o (7 A „ | > c/2} 

+MI/A°^A n -A°f/A n | >c/2}, 

m{|/a„ o C/ An - fx oU Xn \> c/2} = E[Lx n l {lfXn . M>c/2} ] 

by i/ie uniform integrability of (Lx n , n > 1) and </ie continuity of A — > /a. XTie second term tends 
also to zero by the standard use of Lusin theorem and again by the the uniform integrability of 
{Lx n ,n>l). 

Corollary 1. The map A — > 5ua)|?7a] is continuous as an L p (n) -valued map for any p > 1. 

Proof: We know that 

£[[ p (-fc A )|t/ A ] = _L^. 

^A ° Ux 

□ 

Corollary 2. Let Z x {t) be the innovation process associated to Ux, then 

A^ f E[ux{s)\Ux{s)]dZx(s) 
Jo 

is continuous as an L p {ji) -valued map for any p > 1. 
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Proof: We have 

\ogL x oU x = E[u x (s)\U x (s)]dZ x ( S ) + ^^ \E[u x ( S )\U x ( S )}\ 2 ds, 

since the l.h.s. of this equality and the second term at the right are continuous, the first term at the 
right should be also continuous. □ 



Theorem 6. Assume that 

E [ \5{S(K a u' a )K a u' a - K a Vu' a K a u' a + K a u^}\da < oo 
Jo 

for any A > 0. Then the map 

d r 

is a.s. absolutely continuous w.r.to the Lebesgue measure dX and we have 

^L x {w)=L x E[SD x \U x = w], 

where 

D x = 5{K x u' x )K x u' x - K x Vu' x K x u' x + K x u' x ' . 
Proof: Let / be a smooth function on W, using the integration by parts formula as before, we get 

-^E[foU x ] = ±E[foU x 5(K x u' x )] 

= E[(VfoU Xl u' x ) H 5{K x u' x )\ 

= E[{K* x V{f o U x ), u' x ) H d(K x u' x ) + f o U x S(-K x \7u' x K x u' x + K x u'i)} 

= E[fo U x {S(S(K x u' x )K x u' x ) - S(K x \7u' x K x u' x ) + 6(K x u'i)}} . 

Let us define the map D x as 

D x = 5{K x u' x )K x u' x - K x Vu' x K x u' x + K x u' x ' , 

we have obtained then the following relation 

d 2 



hence 

< 



dxi E[foU x ] = E[fL x E[SD x \U x = w]] 



^L x , f>-< -^-L x , f > | A=0 = j X E[f L a E[6D a \U a = w]]da . 



The hypothesis implies the existence of the strong (Bochner) integral and we conclude that 

L' x — L' Q = f L a E[SD a \U a = w]da 
Jo 

a.s. for any A, where L' x denotes the derivative of L x w.r.to A. □ 
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Theorem 7. Define the sequence of functionals inductively as 

= (6DV)K XV ! X + ±DV 



Assume that 



E / \5D^\da < oo 
Jo 



for any n > 1 and A € IR, then A — > L\ is almost surely a C 00 -map and denoting by L^' its 
derivative of order n > 1, we have 

4" +1) H - 4" +1) H = t L a E[8D^\U a = w]da . 

Jo 

4. Applications to the invertibility of adapted perturbations of identity 

Let u 6 L*(ji,H), i.e., the space of square integrable, H- valued functionals whose Lebesgue den- 
sity, denoted as u(t), is adapted to the filtration (T t ,t e [0, 1]) di-almost surely. A frequently asked 
question ire the conditions which imply the almost sure invertibility of the adapted perturbation of 
identity (API) w — > U(w) = w + u(w). The next theorem gives such a condition: 

Theorem 8. Assume that u 6 L^fi, H) with E\p{— Su)] — 1, let u a be defined as P a u, where 
P a = e~ aC denotes the Ornstein- Uhlenbeck semi-group on the Wiener space. If there exists a Ao 
such that 



E / E[p(-5u a )\U x ] E[5{K a u' a )\U a ] 



da 



E [ E[p(-Su a )\U x ] E[S((I H + Vu a )- l Cu a )\U a 
Jo 



da < oo 



for A < Ao, then U is almost surely invertible. In particular the functional stochastic differential 
equation 

dV t (w) = -u(V s (w), s < t)dt + dW t 
V = 

has a unique strong solution. 

Proof: Since u a is an H — C°°-function, cf. [19], the API U a — Iw + u a is a.s. invertible, 
cf.[20 , Corollary 1. By the hypothesis and from Lemma 2 of 20 , (p(— Su a ),a < Ao) is uniformly 
integrable. Let L a and L be respectively the Radon-Nikodym derivatives of U a p and Up w.r. to p. 
From Theorem [31 

L\(w) = L(w) exp / E[S(K a u' a )\U a — w]da 
Jo 

for any A < Aq and also that \E[S(K a u' a )\U a — w]\da < oo almost surely. Consequently 



L x - L= ^exp E[S{K a u' a \U a = w]da - lj L 
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as A — > 0, in probability (even in L 1 ). We claim that the set (L a \ogL a ,a < A ) is uniformly 
integrable. To see this let A £ T, then 

E[l A L a log L a ] = E[1 A o U a log L o U a ] 

= -E[1 A o U a log E[p(-Su a )\U a }] 
< -E[l A o U a log p(-6u a )] 

= E 



1 



1a o U a \ 5u a + ^\u a \ 2 H 

Since (|u a | 2 , ct < X ) is uniformly integrable, for any given e > 0, there exists some 7 > 0, such that 
sup a -E[iB|wa| 2 ] < £ as soon as (i(B) < 7 and this happens uniformly w.r. to B, but as (L a , ct < \q) 
is uniformly integrable, there exists a 71 > such that, for any A £ J 7 , with fJ-(A) < 71, we have 
A t (^<^ 1 ( j 4)) ^ 7 uniformly in a and we obtain E[l A ° J/clitcl^] < e with such a choice of A. For the 
first term above we have 

E[1 A o U a 5u a ] < E[l A L a }^ 2 \\u a \\ L 2 { ^ H} < e 
again by the same reasons. Hence we can conclude that 

lim E[L a logL a ] = ML log L] . 

a— >-0 

Moreover, as shown in (XSJ US], the invertibility of U a is equivalent to 



E[L a logL c 



1 



E[K\h] 



1 



E[\u\l], 



therefore 



E[LlogL] = -E[\u\ 2 H ] 



which is a necessary and sufficient condition for the invertibility of U 



□ 



In several applications we encounter a situation as follows: assume that u : W — > H is a measurable 
map with the following property 

\u(w + h) — u(w)\ H < c\h\ H 

a.s., for any h £ H, where < c < 1 is a fixed constant, or equivalently an upper bound like 
II Vu 1 1 op < c where || • || op denotes the operator norm. Combined with some exponential integrability 
of the Hilbert-Schmidt norm Vu, one can prove the invertibility of U = Iw + u, cf. Chapter 3 of 
|19j . Note that the hypothesis c < 1 is indispensable because of the fixed-point techniques used to 
construct the inverse of U. However, using the techniques developed in this paper we can relax this 
rigidity of the theory: 

Theorem 9. Let U\ = Iw + Xu be an API (adapted perturbation of identity) with u £ lD p ,i(_ff) n 
L 2 (/i,H), such that, for any X < 1, U\ is a.s. invertible. Assume that 

(4.4) E ( p{-8{au))\E[5{{I H +aVu)- 1 u)\U a ]\da < 00. 

Jo 

Then U — U\ is also a.s. invertible. 
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Proof: Let L = L\ be the Radon-Nikodym derivative of U±/j, w.r. to /i. It suffices to show that 

E[L\ogL] = \E\\uW\ 

which is an equivalent condition to the a.s. invertibility of U, cf. [16] . For this it suffices to show 
first that (L x , A < I) converges in L°(ii) to L, then that (L\logL\, A < I) is uniformly integrable. 
The first claim follows from the hypothesis (|4.4j) and the second claim can be proved exactly as in 
the proof of Theorem [5] □ 



5. Variational applications to entropy and estimation 



In the estimation and information theories, one often encounters the problem of estimating the 
signal u\ from the observation data generated by U\ and then verifies the various properties of 
the mean square error w.r. to the signal to noise ratio, which is represented in our case with the 
parameter A. Since we know that (|16j) 



1 f 1 
E\Lx log L\] = -E / \E[ 
1 Jo 



iix(s)\Ux(s)}\ 2 ds, 

the behavior of the mean square error is completely characterized by that of the relative entropy. 
Let 6 denote the entropy of L\ as a function of A: 

0(A) =E[L X log L x ]. 

From our results, it comes immediately that 

d9(X) 



dX 



E[L' x \ogL x ] 
E[L X E[6(K x u' x )\U x = w] log L x ] 
E[E{S{Kxu' x )\Ux\logLxoUx\ 
-E[S(Kxu'x)logE\p(-Sux)\Ux}]. 



Similarly 



d 2 9(X) 
dX 2 



= E 



L'llogLy 



E\L" X log L x + L x E[S(K x u' x )\U x - w} 2 } 
E[E[6D x \U x =w]L x \ogL x + +L x E[S(K x u' x )\U x = w] 2 } 
E[E[SD x \Ux\ logL x o U x + E[S(K x u' x )\U x ] 2 ] . 



In particular we have 



Theorem 10. Assume that 

r-l 

E E\SDx\Ux 



E[ux(s)\Ux(s)]dZx(s) + - 



\E{ux(s)\Ux(s)]\ 2 d; 



< E [E[8{Kxu' x )\U x ] 



for some A = Ao > 0, then there exists an e > such that the entropy is convex as a function of 
X on the interval (Ao — e, A + e). In particular, if Uq = 0, then the same conclusion holds true on 
some (0, e). 
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5.1. Applications to the anticipative estimation. In this section we study briefly the estimation 
of u\(i) with respect to the final filtration U x (l) — a(U x ). 

Theorem 11. Assume that 

E [ L a \E[u' a {s) + 6(u a (s)K a u' Q )\U a }\ p da < oo , 
Jo 

for a p > 1, then, dt-a.s., the map A — > L x E[u x (t)\U x = x] and hence the map A — > E[ii x (t)\U x = x] 
are strongly differentiable in L p (/i) for any p > 1 and we have 

■^-E[u x (t)\U x =x} = E[u' x {t) + 8{u x {t)K x u' x )\U x = x] - E[u x {t)\U x = x]E[8{K x u' x )\U x = x] 
d\ 

dfi x dt-a.s. 

Proof: For a smooth function h on W, we have 

<E[u x (t)\U x =x],hL x > = -^-<E[u x (t)\U x ],hoU x > 

= E[u' x (t)h o U x + ux(t)(Vg o U Xl u' x ) H ] 

= E[E[u' x {t)\U x ]h oU x + ho U x S(u x (t)K x u' x )] 

= E [hL x {x) (E[u' x (t)\U x =x} + E[5{u x (t)K x u' x )\U x = x])] . 

The hypothesis implies that this weak derivative is in fact a strong one in L p (p), the formula follows 
by dividing both sides by L x and by the explicit form of L x given in Theorem [3] □ 

Using the formula of Theorem [11] we can study the behavior of the error of non-causal estimation 
of u x (denoted as NCE in the sequel) defined as 

NCE = E f \ii x (s) — E[u x (s)\U x (l)]\ 2 ds 
Jo 

— E f |u A (*)- E[u x {s)\U x }\ 2 ds 
Jo 

To do this we prove some technical results: 
Lemma 3. Assume that 

(5.5) e[ f K{s) + S(u' a {s)K a u , a )\Pdsda<oo 

Jo Jo 

for some p > 1, for any A > 0, then the map 

A -> L x E[u' x (s)\U x = x] 
is strongly differentiable in L p (d^,, L 2 ([0, 1])), and its derivative is equal to 

L x E[u'l(s) + S(u' x (s)K x u' x )\U x = x] 

ds x djjL-a.s. 
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Proof: Let h be a cylindrical function on W, then, using, as before, the integration by parts formula, 
we get 

^E[L x E[u' x (s)\U x =x]h] = ±E[u' x (s)hoU x ] 

= E[u'l(s) hoU x + hoU x 6(u' x (s)K x u' x )} 

= E [hL x (E[u'l(s) + 6(u x (s)K x u' x )\U x = x])} . 

This proves that the weak derivative satisfies the claim, the fact that it coincides with the strong 
derivative follows from the hypothesis (j5.5[) . □ 



Let us define the variance of the estimation as 

p(X,s)=E[\E[u x (s)\U x (l)}\ 2 ] , 

we shall calculate the first two derivatives of A — > j3(X, s) w.r.to A in order to observe its variations. 
Using Lemma |31 we have immediately the first derivative as 



^(M = E 



E[u x {s)\U x = x]L x 



(5.6) (e[u' x (s) + S(u x (s)K x u' x )\U x =x\- ^E[u x (s)\U x = x]E[5{K x u' x )\U x = x] 

The proof of the following lemma can be done exactly in the same manner as before, namely, 
by verifying first the weak differentaibility using cylindrical functions and then assuring that the 
hypothesis implies the existence of the strong derivative and it is left to the reader: 

Lemma 4. Assume that 

E [ \S(S(K a u' a )K a u' a ) + 5{K a < - K a \7u' a K a u' a )\Pda < oo , 
Jo 

for some p > 1 . Then the map 

A^ L x E[S{K x u' x )\U x = x] 
is strongly differentiable in L p (p) and we have 

■£(L x E[S(K x u' x )\U x = x}) = L X E [5{S(K x u' x )K x u' x )\U x = x] 

+L X E [S(K x u'i - K x Vu' x K x u' x )\U x = x] . 
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Combining Lemma [3] and Lemma HI and including the action of L\ , we conclude that 



/3"(A) 



E\E[ul + 5(u' x K x u' x )\U x }E[u x ( S )\U x } 
+E [e[u' x (s)\U x ] (e[u' x (s) + 5(ux(s)Kxu' x )\F x \ 
-E[u x (s)\U x }E[S(K x u' x )\Ux]" 



-E 



E[S{u'i(s)Kxu'x - u'x(s)KxVu'xKxu'x} 



+S{ux(s)Kxu'i + 5{ux(s)Kxu'\)Kxu x }\Ux}E[ux(s)\Ux} 



-E 



E[S(u x {s)Kxu\)\Ux}[E[u' x {s) + 6(ux(s)K x u' x )\Ux\ 
E[u x (s)\U x ]E[S(K x u' x ]\U x f 



-E 



E[u x (s)\U x ] (E[u' x (s) + S(u x (s)K x u' x )\U x ] - E[u x {s)\U x }E[S(K x u' x )\U x }) E[5{K x u' x )\U x ] 
E[E[u x (s)U x ] 2 {E[6(S(K x u' x )K x u' x + K x u'i - K x Vu' x K x u' x )\U x ]} 
Assume now that A — > u x is linear, then a simple calculation shows that 

P"{Q)=E[\u{ S )\ 2 ], 

hence the quadratic norm of the non-causal estimation of u, i.e., the function 

X^E f \E[u x (s)\U\{l)}\ 2 ds 
Jo 

is convex at some vicinity of A = 0. 

5.2. Relations with Monge-Kantorovich measure transportation. Since La log La € L 1 (/i), 
it follows the existence of </>a € ID24, which is 1-convex (cf. [4]) such that (I\y + V0a)m = Ex ■ 
(i.e., the measure with density La), cf. [5]. From the L p -continuity of the map A^Ia and from the 
dual characterization of the Monge-Kantorovich problem, [3Tj , we deduce the measurability of the 
transport potential (f>x as a mapping of A. Moreover there exists a non-causal Girsanov-like density 
Aa such that 



(5.7) 

/^-a.s., where Aa can be expressed as 



A A La o T x 



A a = J(Ta)cx P (--iVfolt, 

where T x — > J{Tx) is a log-concave, normalized determinant (cf.[6]) with values in [0, 1]. Using the 
relation (|5.7p . we obtain another expression for the entropy: 

E [L A log L A ] = E[\ogL x oT x ] 
= -L[logA A ] 

= E -log J(T X ) + 'cj> x \% 
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Consequently, we have 



1 

2 E 



\E[u x (s) \U x (s)]\ 2 ds 



= E - log J(T A ) 
= £[-logJ(T A )] + ^(M,iA-M), 



where dn(p,,Lx ■ n) denotes the Wasserstein distance along the Cameron-Martin space between the 
probability measures fi and L\ ■ \x. This result gives another explanation for the property remarked 
in about the independence of the quadratic norm of the estimation from the filtrations with 
respect to which the causality notion is defined. Let us remark finally that if 

d H (p,L\ ■ pb) = 

then L\ = 1 /i-almost surely hence E[u\(s) | U\(s)] = ds x dp-a,.s. Let us note that such a case 
may happen without having u\ — /i-a.s. As an example let us choose an API, say K\ = I\y + k\ 
which is not almost surely invertible for any A <E (0, 1]. Assume that E[p(— 6k\)] = 1 for any A. We 
have 

dKxpL 



d/.i 



p(-5m\) 



for some to a £ L^fj,, H), define M\ = Iw + m\, then U\ = M\ o K\ is a Brownian motion and an 
API, hence (cf. |17| ) it should be equal to its own innovation process and this is equivalent to say 
that E[u\{s) | U\(s)] = ds x dfji-a.s. 



6. Applications to Information Theory 

In this section we give first an extension of the results about the quadratic error in the additive 
nonlinear Gaussian model which extends the results of [U [9j [101 QJ] in the sense that we drop a basic 
assumption made implicitly or explicitly in these works; namely the conditional form of the signal 
is not an invertible perturbation of identity. Afterwards we study the variation of this quadratic 
error with respect to a parameter on whose depends the information channel in a reasonably smooth 
manner. 

Throughout this section we shall suppose the existence of the signal in the following form: 

U(w, m) = w + u(w, m) 

where m runs in a measurable space (M, M.) governed with a measure v and independent of the 
Wiener path w, later on we shall assume that the above signal is also parametrized with a scalar 
A £ 1R. We suppose also that, for each fixed to, w — > U(w, m) is an adapted perturbation of identity 
with E^[p{~8u{-,m))] = 1 and that 

/ / \u s (w, m)\ 2 dsdisd/i < oo . 
Jo JwxM 

In the sequel we shall denote the product measure /i <g> v by 7 and P will represent the image of 7 
under the map (w, to) — > (U(w, m), to), moreover we shall denote by Pjj the first marginal of P. 

The following result is known in several different cases, cf. [TJ [51 [TU1 E], and we give its proof in 
the most general case: 
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Theorem 12. Under the assumptions explained above the following relation between the mutual 
information I(U, m) and the quadratic estimation error holds true: 

I(U,m)= f log J P dP=\E 1 f (\E^u s (w,m)\U s (m)}\ 2 -{E^U^Ads, 
JwxM dP v ®dv 2 ' J \ J 

where (U s (m), s € [0, 1]) is the filtration generated by the partial map w — > U(w,m). 



Proof. Let us note that the map (s,w,m) — > E fl [f s \l4 s (m)] is measurable for any positive, optional 
/. To proceed to the proof, remark first that 

dP dP dj 



(6.8) 



dPjj (g> dv c?7 dPu <8> dv 



-l 



dj _ dp, ® dv _ f dP v 
dPu (g) dv dPu ®du~\d^ 

since Pjj ~ fi. Think of w — > U(w,m) as an API on the Wiener space for each fixed m G M. 
The image of the Wiener measure fj, under this map is absolutely continuous w.r. to /i; denote the 
corresponding density as L(w,m). We have for any positive, measurable function / on W x M 

E P [f] = E^foU] 

f(U(w, m), m)dh'( j m)dfi(w) 



WxM 

dll (-, m)/i 



M 

E-r [fL] , 



/- 



dfi 



dv(m) 



hence (w, m) — > L(w, m) is the Radon-Nikodym density of P w.r. to 7. From [16) we have at once 

1 r 1 

E^[L(;m)logL(;m)] = -E^ / \E^[u s (-, m)\U s {m)]\ 2 ds . 

1 Jo 

Calculation of dPu/dfi is immediate: 

L = — — (w) = [ L(w,m)dv(m). 



dji 

Moreover from the Girsanov theorem, we have 



E 7 [foUp(-Su(;m))] = E^[f] 

for any / £ Cb(W). Denote by Ut the sigma algebra generated by (U s : s < t) on W x M. It is easy 
to see that the process Z = (Z t , t e [0, 1]), defined by 

ft 



Z t = Ut(w,m) - / S 7 [u s |W s ]ds 
Jo 



is a 7-Brownian motion and any (Ut, t € [0, 1])- local martingale w.r. to 7 can be represented as a 
stochastic integral w.r. to the innovation process Z, cf. [7]. Let p denote 

(6.10) p = exp(^-^ E 1 [u s \U s ]dZ s ~ 1 J \E 7 [u s \U s ]\ 2 ds^ 

Using again the Girsanov theorem we obtain the following equality 

E, ( [f o Up] =E 1 [fo Up(-Su(w, to))] 
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for any nice /. This result implies that 



E 1 [p{-5u)\U] = p 

7-almost surely. Besides, for nice / on W, 

E Pu [f] = E 1 [foU]=E 1 [fL]=E 1 [fL] 
= E n [f 0UL0U p(-8u)} 
= E y [foULoUp] 

which implies that 

to Up= 1 

7-almost surely. We have calculated all the necessary ingredients to prove the claimed representation 
of the mutual information I(U,m): 

G?7 



I(U, m) = E P 

— Ep 

= Ey 

= E-y 

= E-y 



dj dPjj <g) dv 

' dP , dj 
log h log 

c?7 dPu <S> dv 

dp. dp] r dPu 

-j- log — - E P log 

07 07 J 

dP~\ 

— log— -£ P[7 lo. 
07 dj 

dP lQ ^]_ E 
d'f d7 ^ I dp, dp 



dp 
dPu 



dp 



dPu . di^ 
lor 



i? 7 [i log L] - £7 



1 TT 

log — — o U 

dp 



1 f 1 

= -E 7 J \Ep[u B (w,m)\U a (m)]\ 2 ds-E y [-]ogp] 
and inserting the value of p given by the relation ()6.10j) completes the proof. 

□ 

Remark: The similar results (cf. [2[TUJ[TT]) in the literature concern the case where the observation 
w — > U(w, m) is invertible 7-almost surely, consequently the first term is reduced just to the half of 
the L 2 (p, 7J)-norm of u (cf. [16]). 
The following is a consequence of Bayes' lemma: 

Lemma 5. For any positive, measurable function g on W x M, we have 

1 



emu] 



LoU \Jm 



L(x, m)E^ g \ U(-,m) = x 



dv(rn 



^-almost surely. In particular 

E 1 [g\U = x] 
Pu and p-almost surely. 



L(x) Jm 



L(x, m)E^ g | U(-,m) = x dv{m) 
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Proof. Let / e Cb{W) and let g be a positive, measurable function on W x M. We have 
E^gfoU] = [ E„[E^g\U{-,m)]foU{-,m)]dv{m) 

JM 



IM JW 



L(w,m) E^[g \ U(-,m) = w] f(w)d[i(w)dv(m) 



= \ f( w )\ L(w,m)E fl [g\U(-,m) =w]du(m)) dfj, 

JW \JM J 

= I TTt/M(7 L ^,m)E^g\U{-,m) = w)dv{m)]d l jL 
Jw L{w) \JM J 

— — / oU ( [ L(w,m)E fl {g \ U(-,m) = w] dv{m) 
o U \Jm 



= E. 



□ 

From now on we return to the model U x parametrized with A <E 1R and defined on the product space 
W x M; namely we assume that 

U\{w, m) =w + u\(w, m) 

with the same independence hypothesis and the same regularity hypothesis of A — > u x where the 
only difference consists of replacement of the measure \x with the measure 7 while defining the spaces 

Lemma 6. Let L\(w) denote the Radon-Nikodym derivative of Pu x w.r. to \i. We have 



L\(w) =L (w)exp / E 7 S(K a u' a )\U a = w 



da 



H-almost surely. 

Proof. For any nice function / on W, we have 



On the other hand 



d\ 



E^foUx] = E 7 [foU x S(K x u' x )} 



= E 7 [foU x E 7 [6(K x u' x )\Ux]] 

= E 1 [fL x (x,m)E 1 [5{K x u' x )\U x = x]] 

= E lx [fL x E 1 [5{K x u' x )\U x = x]]. 

Remark: Note that we also have the following representation for L x (w 7 m): 



□ 



L x (w,m) = L (w,m)exp / E^ 5(K a u' a (-,m))\U a (- 

Jo l 



m) = w 



da 



Lemma 7. Let A — > r(A) be defined as 



T{\)=E 1 {L x \ogL x ], 
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where L\(w) = J M Lx(w,m)di>(m) as before. We have 



dr(X) 
dX 



E^[S(K x u' x )\U x ]logL x oUx 



where px is given by i6.10\) as 

px = exp 
Besides, we also have 



E 7 [ux(s)\Ux(s)}dZx(s) 



|£ 7 [« A (s)|W A (s)]| 2 ds 



d 2 r(X) 
dX 2 



whe 



= E 1 [E^SDxlUxK- \ogpx) + E^[S(Kxu'x)\Ux] 



Dx = 5{K x u'x)Kxu'x + -jzKxu'x ■ 
dX 



Proof. The only thing that we need is the calculation of the second derivative of L A : let / be a 
smooth function on W, then, from Lemma 



d 2 
1\ 



^[foUx] 



d 



E y [foUxS(Kxu'x)} 



foU^S( 6(Kxu'x)Kxu'x + ^(KW X ) 



E 7 [foU y SDx] 

E 1 [f{x)E 1 [8Dx\Ux = x] Lx(x)]. 



□ 



As an immediate consequence we get 
Corollary 3. We have the following relation: 



d 2 , 



E^ [6(D\(-,m))\ Ux (m)] (- log E„ \p(-Sux (■ , m) ) | U x (m)] 



+E f ,[S(Kxu'x(-,m))\Ux(m)f 



E y [5(Dx)\Ux\(-logpx)+E J [S{Kxu'x)\Uxf 
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